This invention relates to self-referenced tracking.
Virtual reality (VR) systems require tracking of the orientation and position of a user""s head and hands with respect to a world coordinate frame in order to control view parameters for head mounted devices (HMDs) and allow manual interactions with the virtual world. In laboratory VR setups, this tracking has been achieved with a variety of mechanical, acoustic, magnetic, and optical systems. These systems require propagation of a signal between a fixed xe2x80x9csourcexe2x80x9d and the tracked xe2x80x9csensorxe2x80x9d and therefore limit the range of operation. They also require a degree of care in setting up the source or preparing the site that reduces their utility for field use.
The emerging fields of wearable computing and augmented reality (AR) require tracking systems to be wearable and capable of operating essentially immediately in arbitrary environments. xe2x80x9cSourcelessxe2x80x9d orientation trackers have been developed based on geomagnetic and/or inertial sensors. They allow enough control to look around the virtual environment and fly through it, but they don""t enable the xe2x80x9creach-out-and-grabxe2x80x9d interactions that make virtual environments so intuitive and which are needed to facilitate computer interaction.
In one aspect, in general, the invention provides a new tracking technique that is essentially xe2x80x9csourcelessxe2x80x9d in that it can be used anywhere with no set-up of a source, yet it enables a wider range of virtual environment-style navigation and interaction techniques than does a simple head-orientation tracker, including manual interaction with virtual objects. The equipment can be produced at only slightly more than the cost of a sourceless orientation tracker and can be used by novice end users without any knowledge of tracking technology, because there is nothing to set up or configure.
In another aspect, in general, the invention features mounting a tracker on a user""s head and using the tracker to track a position of a localized feature associated with a limb of the user relative to the user""s head. The localized feature associated with the limb may include a hand-held object or a hand-mounted object or a point on a hand.
In another aspect, in general, the invention features mounting a sourceless orientation tracker on a user""s head and using a position tracker to track a position of a first localized feature associated with a limb of the user relative to the user""s head.
In another aspect, in general, the invention features tracking a point on a hand-held object such as a pen or a point on a hand-mounted object such as a ring or a point on a hand relative to a user""s head.
In another aspect, in general, the invention features using a position tracker to determine a distance between a first localized feature associated with a user""s limb and a second localized feature associated with the user""s head.
In another aspect, in general, the invention features a position tracker which includes an acoustic position tracker, an electro-optical system that tracks LEDs, optical sensors or reflective marks, a video machine-vision device, a magnetic tracker with a magnetic source held in the hand and sensors integrated in the headset or vice versa, or a radio frequency position locating device.
In another aspect, in general, the invention features a sourceless orientation tracker including an inertial sensor, a tilt-sensor, or a magnetic compass sensor.
In another aspect, in general, the invention features mounting a display device on the user""s head and displaying a first object at a first position on the display device.
In another aspect, in general, the invention features changing the orientation of a display device, and, after changing the orientation of the display device, redisplaying the first object at a second position on the display device based on the change in orientation.
In another aspect, in general, the invention features determining the second position for displaying the first object so as to make the position of the first object appear to be fixed relative to a first coordinate reference frame, which frame does not rotate with the display device during said changing of the orientation of the display device.
In another aspect, in general, the invention features displaying the first object in response to a signal from a computer.
In another aspect, in general, the invention features mounting a wearable computer on the user""s body, and displaying a first object in response to a signal from the wearable computer.
In another aspect, in general, the invention features displaying at least a portion of a virtual environment, such as a fly-through virtual environment, or a virtual treadmill, on the display device.
In another aspect, in general, the invention features displaying a graphical user interface for a computer on the display device.
In another aspect, in general, the invention features first object being a window, icon or menu in the graphical user interface.
In another aspect, in general, the invention features the first object being a pointer for the graphical user interface.
In another aspect, in general, the invention features changing the position of the first localized feature relative to the position tracker and, after changing the position of the first localized feature, redisplaying the first object at a second position on the display device determined based on the change in the position of the first localized feature.
In another aspect, in general, the invention features displaying a second object on the display device, so that after changing the position of the first localized feature, the displayed position of the second object on the display device does not change in response to the change in the position of the first localized feature.
In another aspect, in general, the invention features determining the second position so as to make the position of the first object appear to coincide with the position of the first localized feature as seen or felt by the user.
In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame in response to a signal being received by the computer.
In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame in response to a change in the position of the first localized feature.
In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame in response to a signal representative of the location of the user.
In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame in response to a signal representative of a destination.
In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame in response to a signal representative of a change in the user""s immediate surroundings.
In another aspect, in general, the invention features changing the orientation of the first coordinate reference frame is changed in response to a signal representative of a change in the physiological state or physical state of the user.
In another aspect, in general, the invention features redisplaying the first object further comprises changing the apparent size of the first object according to the change in position of the first localized feature.
In another aspect, in general, the invention features mounting a portable beacon, transponder or passive marker at a fixed point in the environment and determining the position vector of a second localized feature associated with the user""s head relative to the fixed point.
In another aspect, in general, the invention features determining the position vector of the first localized feature relative to the fixed point.
In another aspect, in general, the invention features mounting a sourceless orientation tracker on a second user""s head and determining the position of a localized feature associated with the body of the second user relative to the fixed point.
In another aspect, in general, the invention features determining the position vector of a second localized feature associated with the user""s head relative to the fixed point without determining the distance between the second localized feature and more than one fixed point in the environment.
In another aspect, in general, the invention features displaying the first object at a third position after displaying the first object at the third position, changing the orientation of the display, and after changing the orientation of the display, continuing to display the first object at the third position.
In another aspect, in general, the invention features the first object being a window in a wraparound computer interface.
In another aspect, in general, the invention features redisplaying the changed position of the first localized feature not being within the field of view of the display when the first object is redisplayed.
In another aspect, in general, the invention features displaying the first object at a position coinciding with the position of the first localized object when the first localized object is within the field of view of the display.
In another aspect, in general, the invention features positioning the first localized feature at a first point positioning the first localized feature at a second point and calculating the distance between the first point and the second point.
In another aspect, in general, the invention features determining a position vector of the first localized feature relative to a second localized feature associated with the user""s head and modifying the position vector based on an orientation of the user""s head.
In another aspect, in general, the invention features setting an assumed position for the user""s head in a coordinate system and setting a position for the first localized feature in the coordinate system based on the assumed position of the user""s head and said position vector.
In another aspect, in general, the invention features measuring the orientation of the user""s head relative to a fixed frame of reference.
In another aspect, in general, the invention features setting a virtual travel speed and direction for the user modifying the assumed position for the user""s head based on the user""s virtual travel speed and direction.
In another aspect, in general, the invention features mounting on the head of a user a three degree of freedom orientation tracker for tracking the orientation of the head, and a three degree of freedom position tracker for tracking the position of a first localized feature on the user""s limb relative to a second localized feature on the user""s head, computing a position vector for the first localized feature relative to the second localized feature, determining a rotation matrix based on information received from the rotation tracker, and transforming the position vector into a position vector for a fixed frame of reference based on the rotation matrix.
In another aspect, in general, the invention features using an acoustic or radio frequency position tracker to track a position of a first localized feature associated with a limb of the user relative to the user""s head.
In another aspect, in general, the invention features mounting a video camera on the back of the user""s head and displaying an image generated by the video camera in a portion of a display device mounted on the user""s head.
In another aspect, in general, the invention features mounting a first inertial sensor on a user""s head, mounting a second inertial sensor elsewhere on the user""s body or in an object held by the user, and tracking the position of one inertial sensor relative to the other.
Some embodiments of the invention include sensing data at the first and second inertial sensors and using the sensed data to track the position of one inertial sensor relative to the other, tracking the position of the inertial sensor is done without reference to any signal received from a source not mounted on or held by the user and correcting the drift of the relative position or orientation of the second inertial sensor relative to the first inertial sensor by measurements between devices on the user""s head and devices elsewhere on the users body.
Among the advantages of the invention are one or more of the following. The device is easy to don, can track both head and hand, adds no new cables to a wearable computer system, works anywhere indoors or outdoors with no preparation, and is simpler than alternatives such as vision-based self-tracking.